Composable Markov Building Blocks

In sensor data management, it is often useful to collect statistics about state changes of an observed phenomenon. The most basic case is to keep track of how often certain transitions, e.g. from state c to d, have occurred. In our research, we consider a discrete state space and timeline. For example, in the above figure, the state space corresponds to actual two-dimensional space and is divided into discrete granules. While an object moves through this space, its position (state) in terms of these granules is known at each point in time. Counting the transitions between each pair of adjacent granules in a certain stretch of time (including self-transitions, points in time where the object stays put) provides exactly the information needed to construct a first-order Markov model of the object’s behaviour. This model can be used to predict future behaviour or fill inmissing readings. However, in our research we assume that the state space is divided into clusters, and that it is impossible to track an object across a cluster boundary, due to technical or organizational issues. For example: